How can NIH strengthen stewardship through the development of robust measures for assessing the impact of its investments?

The HHS National Institutes of Health and the Office of Management and Budget hosted a researcher interest meeting on May, 30, 2024. Click herefor the slides from that meeting. A question and answer document is also available here.

NIH is the primary federal agency for leading, conducting, and supporting biomedical and behavioral research.  NIH provides financial support to researchers throughout our nation and the world through its 27 Institutes and Centers (ICs) and the Office of the Director. NIH conducts and supports biomedical, behavioral, and social science research across a broad spectrum of scientific disciplines and approaches in pursuit of its mission to seek fundamental knowledge about the nature and behavior of living systems and to use that knowledge to enhance health, lengthen life, and reduce illness and disability. NIH research focuses on both ongoing and emerging public health needs. As NIH identifies these needs, it uses different scientific approaches to understand the basic causes and mechanisms of disease, find new ways of identifying, preventing, and curing disease processes, and bring these new interventions into common practice for the public’s benefit.

NIH is seeking to engage with the scientific community to study NIH processes and programs to further strengthen the evidence base that supports NIH decision-making.  The proposed research efforts focus on strengthening the agency’s evidence-building capabilities in order to improve the public’s and NIH’s own understanding of its impacts. Proposed research should address aspects of one or more of the following questions:

1. How can NIH improve on identifying desired outcomes and measuring impact related to its mission?

a.Beyond bibliometric measures, what are the indicators of scientific success?

b. What approaches can be used to capture successful/impactful scientific strategies and new tools and methods, and are these approaches scalable?

c. What measures can NIH use to capture both incremental knowledge gains and failures that ultimately contribute to scientific success?

d. What approaches can NIH use to measure impact of different categories of science (e.g., basic, translation, clinical) and the technology and operations used to support the science?

e. Are there better ways for NIH to trace dissemination of clinical research findings into:

                                                        i. Care received by patients?

                                                      ii.Use by clinical communities and healthcare providers?

                                                    iii.Use by public health agencies other than NIH?

f. How can NIH capture the economic impact of its outcomes? How does NIH-funded research lead to increased productivity and give rise to new industries?

g. How can these approaches best inform strategies for funding research, resources, and training?

2. Are there methods that NIH can use to better predict and identify scientific opportunities (e.g., the emergence of gene editing technology)?

3. Are there approaches that could inform NIH funding decisions by measuring scientific quality, rigor, and reproducibility?

4. What evidence can NIH use to inform its efforts to optimize its investment in recruiting, training, and sustaining a diverse U.S. biomedical, behavioral, and social sciences research workforce?

a. What data and methods may be used to capture trainee career outcomes?

b. How can NIH evaluate its efforts to expand and diversify the U.S. biomedical, behavioral, and social sciences research workforce through engagement activities?

5.What evidence does NIH need to improve the clinical research ecosystem?  What would inform a re-envisioning of the clinical trials system to maximize quality, participant experience, accessibility, timeliness, and impact on clinical care?

a.How can NIH ensure there is equitable representation of the U.S. populations in its funded clinical research so health disparities are not compounded in underrepresented populations?

6. What evidence could inform steps NIH can take to ensure progress on overcoming health disparities and strengthen partnerships with underserved communities and practitioners?

7. How should NIH assess risk in its research portfolio?  What is the right amount of risk for NIH to accept as a steward of public funds? 

a.Does the NIH funding system foster sufficient risk-taking to encourage researchers to explore high-risk research?  If not, are there ways to test novel funding approaches that could be implemented at scale? 

Data on NIH expenditures and the results of NIH supported research are available within Research Portfolio Online Reporting Tools (RePORT),[1] which provides access to reports, data, and analyses of NIH research activities. If researchers have specific suggestions for relevant administrative data to help inform their proposed project, NIH can assess whether such data exists and can be shared.

[1]https://report.nih.gov/